Recognizing and Interpreting Gestures within the Context of an Intelligent Robot Control Architecture

In this paper we describe a stereo vision system that can recognize natural gestures and an intelligent robot control architecture that can interpret these gestures in the context of a task The intelligent control archi tecture consists of three layers The top layer delib erates via a state based hierarchical non linear plan ner The bottom layer consists of a suite of reactive skills that can be con gured into synchronous state machines by the middle layer The middle layer im plemented in the RAPs system serves to mediate be tween the long range deliberation of the top layer and the continuous activity of the bottom layer It is in the middle layer that we are investigating context focused deictic gesturing for human robot interaction When directed in the context of di erent RAPs human ges tures can be interpreted by the system di erently for di erent tasks This work shows that a our archi tecture designed to support perception and action can also support other forms of communication and b task contexts can act as resources for communica tion by simplifying the interpretation of communica tive acts